Digitali image 
Processing 


Instructor: Xiaoli Zhao 


* Objectives: This 3-credit course provides 
education and training to equip students 
to: 


" Explain various image representations, 
human visual perception, color space, and 
standards. 

" Write computer programs to perform image 
filtering, enhancement, restoration, and 
transform. 

" Perform various image compression and 
segmentation using various techniques. 

" Implement algorithms to solve real-world 
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Prerequisites 


E Text book: 


— Rafael C. Gonzalez and Richard E. Woods, 
Digital Image Processing, 4rd Ed. Pearson, 
2020 


— Rafael C. Gonzalez and Richard E. Woods, 
Digital Image Processing, 3rd Ed. Pearson, 
2007. 

= Prerequisites by Topic: 

— Discrete Time Signal Analysis, familiarity with 

PYTHON 


Course Contents 


Lecture 1: Image Representations, Visual 
Perception, Color Space (6 hours) 


Lecture 2: Spatial Domain Image Enhancement 
(10 hours) 


Lecture 3: Spatial Domain Image Filtering (10 
hours) 


Lecture 4: Frequency Domain Image Filtering 
and Enhancement (10 hours) 


Lecture 5: Edge Detection and Segmentation (6 
hours) 


Lecture 6: Mathematical Morphology (4 hours) 


Final Exam & Review: (2 hours) 


Grading 


= Attendance and Special Performance: 
20% 

= Presentation, HWs and Processing 
Labs: 40% 

a Final Exam.: 40% 


= Special bonus to students with active 
class participation and biggest exam 
improvement 


1. Digital Image 
Representations 


Chapter 1 (Introduction) 

Chapter 2 (2.1-2.3 Visual Perception) 
Section 2.4 (Sampling and Quantization) 
Section 6.2 (Color Models & Image Formats) 


LIEIMENLS OF iImage/video 
Processing 


Applications: multimedia, 
medical, surveillance, TV, 
video conferencing, video-on- 
demand,... 

Acquisition: sensing and 
digitizing (film, CCD, digital 
Still camera, video camera, 
camcorder, Kinect) 

Storage: disks, tapes, ... 
Processing: software on 
general-purpose or dedicated 
computers, hardware 

boards, ... 

Communication: wireless, 
Internet, ... 6 
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Optical Representation of Eye 
Viewing 


Point C is the optical 
center of the lens 
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Structure of Human Eyes 


= 3 membranes: Cornea & 
Sclera (outer cover), 
Choroid (ciliary body & 
iris), and Retina 

m Light receptors over retina: 
cones (6-7 million) & rods 
(75-150 million) 

= Cones: located primarily in 
the central portion of Retina 
called Fovea, highly 
sensitive to color, can 
resolve fine details ET 

= Rods give an overall picture | / 
of field of view, not involved | 
in color vision, sensitive to 


Ciliary muscle 


Visual axis 


Vitreous humor 


Eyes and Colors 


a Rods (— ) detect movement, shape, 
and textures (scotopic vision, dim- 
light vision), but unable to convey 
details because they are connected in 
bunches. 


E Cones (6.5 x ), only operate in 
medium to high 
levels of illumination, find details of 
colors and shades (photopic vision, 
bright-light vision). 


= Thomas Young (1802): three types of 
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Absorption (arbitrary units) 


Absorption of Light in Cones 
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Color Fundamentals 
= Chromatic light spans EM 


spectrum from 360 nm to 770 nm, 


(more sensitive to green light, 
then red, then blue). 

= Three basic quantities of 
chromatic light: radiance, 
luminance, and brightness. 


— Radiance (watts): total energy flow 
from the light source. 

— Luminance (lumens): amount of 
energy an observer perceives. 


— Briahtnace- cinhiactiva daccrintar 


A Simplified Human Visual 
Perception 


= The perceived intensity f(x,y) 
consists of light reflected from 
objects: 


f(x,y) = I(x, y) r(x,y) 


-0 < f(x,y) < infinity, 

-0 < I(x,y) (illumination) < 
infinity 

-0 < r(x y) (reflectance) < 1 
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Color Spectrum 


Plate 1. Color spectrum seen by passing white light through a prism 
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Light and Pigments 


Aiditive (light) Subtractive (pain) 


Additive and subtractive color combinations 


Primary & Secondary Primary & Secondary 
Colors of Light Colors of Pigments 14 


Primary Colors of Light 


m Owing to the structure of the human eye, 
all colors are seen as various combinations 
of the three so-called primary colors: Red 
(R), Green (G), and Blue (B). A proper 
combination of these three primaries 
produces white color. 


= Commission Internationale de 
l'Eclairage (CIE) -- International 
Commission on Illumination adopted three 
primary colors: R=700 nm, G=546.1 nm, 
B=435.8 nm. 

a Secondary colors of light: Magenta (red 
plus blue), Cyan (green plus blue), and 
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Primary Colors of Pigments 


Primary Colors of Pigments (Colorants): Cyan 
(G-B), Magenta (R-B), Yellow (R-G). They are 
also called “subtractive primaries”. A proper 
combination of these three primaries produces 
black color. 


A subtractive system used by printers for the 
rendering of colors with ink. 

By depositing three color pigments on white 
papers (such as ink-jet printers), each absorbs 
its complementary light color, i.e., cyan 
absorbs red, magenta absorbs green, and 
yellow absorbs blue. By increasing yellow ink, 
the blue decreases. 


The secondary colors of pigments are red, 
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Additive & Subtractive Color 
Systems 


m Additive: colors are created by adding colors 
to black to create new colors. The more 
added, the closer to white. 


= Additive color environments are self- 
luminous. 


= Subtractive: primary colors are subtracted 
from white to create new colors, the more 
taken away, the closer to black. 

a Any color image reproduced on paper is an 
example of subtractive color system. 
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RGB Color System 


= The most widely used system (additive). 
Each pixel is represented as a triplet, 
e.g., Red (255,0,0), Yellow (255,255,0), 
Green (0,255,0), Cyan (0,255,255), Blue 
(0,0,255), Magenta (255,0,255). 


= (0,0,0) denotes black, and 
(255,255,255) denotes white. 
(63,63,63), (127,127,127), etc, 
represent different shades of gray. 


RGB 24-Bit Color Cube 


(0,0,1) 


Blue 


Cyan 


Magenta 


| 
| 
| 
1 


. 
4 
+ 


Black L NR ai, ŻE (0, 1.0) 
Z 


Green 


Red | Yellow 


| 
IURE 6.8 RGB 24-bit color cube. 
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Digitization of Signals 


— Sampling: from continuous time to discrete time 
— Quantization: from continuous valued to discrete 


valued 


Quantization levels 


Quantization levels 
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= f(x, y): (x, y) denote 


A Color Photo 


f(x,y) 


the spatial 
coordinates. In a 


digital image”, (x,y) FAR BY E -n nun L 


is a discrete picture | i AI Agi 
element (pixel or da pa 
pel). For a gray- 
scale image, f() at any 
point (x,y) is the 
brightness (or gray- 
level) of the image. 
For color images, 
each pixel contains 3 
numbers representing 
2 color comnnnante 
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2-D Sampling (Spatial Domain) 


Digital Image Representation 


Origin FIGURE 2.18 
0 1 2 3: «++ N- 1 Coordinate 
0 seer Se [ep convention used 
1 » in this book to 
2 |. 5 4 w è represent digital 
images. 
M-1 1° $ & E & e & o & ais % 
| One pixel =F fx y) 
X 


many other books/papers, x-axis is in the horizontal direetic 


2-D Sampling (Spatial Domain) 


Illumination (energy) 


af | \ source 


Output (digitized) image 


Imaging system 


(Internal) image plane 


Scene element 


= cde 


FIGURE 2.15 An example of the digital image acquisition process. (a) Energy (“illumination”) source. (b) Ane 
ement of a scene. (c) Imaging system. (d) Projection of the scene onto the image plane. (e) Digitized image. 


Resolution 


64 


512 


1024 


FIGURE 2.19 A 1024 x 1024, 8-bit image subsampled down to size 32 X 32 pixels. The number of allowable 
gray levels was kept at 256. 
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FIGURE 2.25 Top row: images zoomed from 128 x 128, 64 X 64, and 32 X 32 pixels to 1024 x 1024 pixels, 
using nearest neighbor gray-level interpolation. Bottom row: same sequence, but using bilinear interpolation. 


esolution (in dpi) 


ab 
cd 


FIGURE 2.20 Typical effects of reducing spatial resolution. Images shown at: (a) 1250 
dpi, (b) 300 dpi, (c) 150 dpi, and (d) 72 dpi. The thin black borders were added for 
clarity. They are not part of the data. 
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Quantization 


f(x) 


* Review binary number system: 
Conversions between binary and decimal number system: 
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Example of Quantization 


‘ontinuous. 
Discrete 


ORIGINAL 
SAMPLE 


RECONSTRUCTION LEVELS 
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QUANTIZED 
SAMPLE 


Quantization 


Example: 


S: 
Quantization: 


Quantization step-size Q=2: 


Quantization Levels (Q): 
Inverse quantization (x2): 
Quantization error 


Quantization step-size Q=4: 


Quantization Levels (Q): 
Inverse Quantization (x4): 
Quantization error 


01234567 (3 bits) 


S/2, truncate 
00112233 (2 bits) 


00224466 
01010101 


S/4, truncate 
00001111 (1 bit) 


00004444 
01230123 
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Quantization Levels 


DECISION LEVELS 
do d, d, d, di_, d, d; d,., d 
La La | a [o eee [af ese, al 
< Ta E Fa E r; Toca Po 
RECONSTRUCTION LEVELS 

(a) LINE REPRESENTATION 


RECONSTRUCTION 
LEVEL 


OUTPUT 


DECISION LEVEL 
INPUT 


(b) STAIRCASE REPRESENTATION 


Quantization 


Output Output 
codewords codewords 


Linear Non-linear 
Quantizer Quantizer 


Input level 


Input level 


Many to one mapping that reduces the number of possible 
signal 


values at the cost of introducing distortions 


Many different ways of implementation (table lookup, divide 
by the 


quantization step-size and round/truncate) 
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FIGURE 2.16 Generating a digital image. (a) Continuous image. (b) A scan line from A to B in the continuous image, 
used to illustrate the concepts of sampling and quantization. (c) Sampling and quantization. (d) Digital scan line. 


LU, 


64, 32, 16, 8, 4, 2 (1-bit) 
levels 
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FIGURE 2.21 
(Continued) 
(e)-(h) Image 
displayed in 16, 8, 
4, and 2 gray 
levels. (Original 
courtesy of 

Dr. David 

R. Pickens, 
Department of 
Radiology & 
Radiological 
Sciences, 
Vanderbilt 
University 
Medical Center.) 
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FIGURE 2.22 (a) Image with a low level of detail. (b) Image with a medium level of detail. (c) Image with a rel- 
atively large amount of detail. (Image (b) courtesy of the Massachusetts Institute of Technology.) 
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Resolution vs. Quantization Levels 


FIGURE 2.23 


Representative NxN pixe Is 
isopreference 


curves for the k bits/pixel 
three types of kN? bits/picture 


images in 
Fig. 2.22. 


< Isopreference curves tend 

to 

become more vertical as 
the 

detail in the image 
increases: 

-> images with a large 
amount 

of detail, only a few gray 
levels 
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Cameraman 


Image Data 


= Binary: O & 1 (1 bit/pixel, mainly line- 
drawing or 
documents) 


= Gray Scale: O (black), shades of gray, 
2º - 1 (white). Typically m=8 (256 
quantization levels) 


= Color: three primary color components, 
e.g. Red (R), 
Green (G), Blue (B). 


= Resolution: 1024x1024, 512x512, 
256x256, 352x240, ... 


Display and Printing with 
Halftoning 


Halftoning: The use of bi-level pixels or dots to create 
the appearance of shades of gray by grouping the pixels 
in patterns to produce the desired shades for printing. 


Different patterns may be used 40 


Dithering 


a Printers of modest dots per inch (dpi) 


have difficulty producing halftone cells 
that are small and still represent many 
gray levels; visually distracting 
patterns appear if regular patterns of 
dots are used. 


Simple random dither: for each 
possible position (where the printer can 
place a black dot), a random number is 
generated; if the number is below the 
gray scale value for that point, then the 
dot is printed. 
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Dithering 


Common Image Formats 


= BMP (Bitmap), GIF (Graphics 
Interchange Format), TIFF 
(Tagged Image Format File), 
JPEG (Joint photographic Expert 
Group), ... 


= Different formats contain 
different header information and 
descriptors, and use different 
data representations and 
compression methods. 


Image Processing 


a Enhancement 

m Restoration 

= Interpolation 

= Compression 

= Object segmentation and 
recognition 

= Content-based indexing/retrieval 

E 
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YIQ, YUV, YCbCr 


m Luminance (Y) and two Chrominance 
components. 


= Goal: for transmission efficiency and for 
maintaining compatibility with 
monochrome TV standard. 


= The human visual system has greater 
sensitivity to changes in luminance than 
to changes in hue or saturation. YIQ 
calls for more bandwidth in representing 
Y, and less for I and Q. Useful for 


YUV, YIQ, YCbCr 


Y=0.299(R-G)+G+0.114(B-G) 


Y 0.299 0.587 0.114 | R C,=0564(B-Y) and C,=0.713(R-Y) 
U| =+0.147 -0.289 0.43 G 
V 0.615 -0.515 -0.10 B Y 0299 0.587 0114][R 
l | | C,|=|-0.169 —0331  0500||G 
E 0500 -0419 -0.081|| B 
Y 0.299 0.587 0.11 R no = — 
I | 710.596 -0.275 -0.32 G G|= 10 _ 03441 sido SA 
Q 0.212 -0.523 0.31 B B| |10 17718 0.0 || C. 


JV is used in European TV broadcasts, 
Q is used in U.S. TV broadcasts. YCbCr used in video codin 


=0.492(B-Y), V=0.877(R-Y) 


‚and C, are scaled and offset versions of U and W. x 


Human Color Perception (HSI/HSV) 


= Human described a perceived color 
in terms of: Hue, Saturation, and 
Intensity (Value). 


= Hue: a subjective measure of color 
using conventional nomenclature: 
such as “red”, “yellow”, or “purple”. 
Normal eye can distinguish among 
200 hues. Colors that contain a hue 
component are called “chromatic 
colors,” otherwise called achromatic 
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HSI/HSV Color Systems (cont.) 


= Saturation: relative purity or the 
amount of “white” light mixed with a 
hue. For example, Pink (50% 
Saturation) has the same perceived 
hue (red), but is a whiter red. 
Lavender is a whiter violet. 


= Intensity (Value): indicates the 
perception of brightness or darkness. 
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HSI/HSV Color Systems (cont.) 


= Hue, Saturation, and Intensity/Value 
(HSI/HSV) are the three main 
characteristics of human perception to 
colors, therefore they offer ideal models 
for image processing systems. 


= Some usefulness of HSI/HSV models: 
design of imaging systems for 
automatically determining the ripeness of 
fruits and vegetables, or inspection of 
quality of finished color goods. 
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HSI Color Triangle & Color Solid 


HSI Color System 


< Hue is defined as the angle with respect 
to the red axis; 

Saturation is represented by the closeness 
to the 

pyramid surface; Intensity is denoted by 
the 

perpendicular distance from the black 
point 


# Red (0,240,120), Yellow (40,240,120), 
Green 
(80,240,120), Cyan (120,240,120), Blue S 
(160 DAN 190) 


RGB to HSI 
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HSI to RGB 


HSI to RGB (cont.) 


HSV 


= HSV is usually 

visualized asa 

cone 

— The bottom 
point of the 
cone is black 

— The top is a 
color wheel 
with white at 
the center and 
strona colors 


A full-color image and its various 
color-space components 


Yellow Black 


Hue Saturation Intensity 


FIGURE 6.30 A full-color image and its various color-space components. (Original image courtesy of Med- 
Data Interactive.) 


High saturation 


Low saturation 
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Brighter image 


Darker image 
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High contrast 


Low contrast 


59 


High resolution 


Low resolution 
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A Tutorial of Python for 
Digital Image Processing 


DOCODDUDDDO scipy OOO0000000000000000000 HOW 
c JO000 


import matplotlib.pyplot as plt 
from scipy import misc 

import scipy 

I = misc.imread('./cc 1.png') 
scipy.misc.imsave('./savel.png', I) 
plt.imshow(I) 

plt.show() 


RUG [| B components 
= R=I(2:,1): 
=+G=l(:,52); 
— B=I(:,:,B); 


